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ABSTRACT 

The Health Consumer Education Program (CEP) vas 
designed to foster "maximum feasible participation" of the people in 
planning, administering, and implementing community health care 
programs through adult education. This report, an analysis and 
evaluation of CEP, describes the p.ogram, presents data on intended 
and current practice, analyzes discrepancies and discusses 
recommendations using data obtained from interviews vitn key CEP 
staff, community participants who served as course coordinators, 
evaluation sheets completed by all course participants, 
questionnaires (Appendixes A and B) mailed to course participants, 
observation of a fev classes, and analysis of documents (proposals, 
progress reports, course outlines). The community participation on 
the steering committees in all four lov^income areas created a broad 
range of learning experiences and articulated the needs of the 
communities. Comparing the summary cf program intent vith the survey 
results from the questionnaires (115 responses, reflecting a 34.6 
percent response rate) and interviews, it vas found that there vere 
no discrepancies betveen the goals and actual practice. 
Recommendations pointed out several weaknesses: the need to recruit 
more community leaders, more emphasis on application of skills and 
knowledge, aore technical assistance for projects, and the 
institution of continuous program evaluation from the inception of 
the program. (JB) 
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lula'oduc tion 



T^vo years ar^o, in July, 1972, the Pro^iram in Conlintiing Education, 
ochool of l\ihlic Health, Columbia University, received a project a^'^'^'nt 
from tlie Ifonlth Resources Administration, l\iblic Health Service, Department 
of HE'/J', to provide educational program? fox* Uew York City health coiisumers 
so that such persons v/ould be better equipped to participate in coiiuaunity 
h.ealth care planning as mandated by federal comprehensive health planning 
le-iislation. 

The philosophy underlying the feaeral initiative in comprehensive 
health planning seems to echo the egalitarian dictum of tne no;v moribund 
V7ar on Poverty: "N!aximum feas-^ble participation" of the people in community 
health care planning. The grant proposal submitted by Columbia Uriiversity 
incorporated an edacatio.tal philcsoph:^ very niiich in harmony v/ith the maximum 
feasible participation thrust of public policy. It was proposed that major 
responsibility for the planning and implementation of educational programs 
for health care consumers would rest with the consumers themselves. The 
University would provide assistance, notably money and staff expertise, but 
the burden of responsibility for assessing educational needs, setting ob- 
jectives, and implementing pro^^rams was to be assumed by the community 
residents themselves. The project proposal, developed in collaboration with 
seven community health planning units (experimental forerunners of community 
health planning boards), set forth the following objectives: 

To help citizens recognize their health rights, gain 
confidence in the importance of their role as consumer 
representatives, learn ';.o articulate the health needs 
of their community and develop both planning and 
administrative skills. 
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The proiior.t report ia an atialysirs and eYalnahion of this unasnal ex- 
pori'n-^nt in coiiuiranity education. 

r^/alurif. Ion Design 

This evaluaMon is largely an ex po?^t facto analysis of the Health 
Consumer lodiication Pi'ojram (CEP). ?\inrlG vrere not available for a two-year 
continuous evalu?.tion, which all parties a^ree vrould have been highly 
der;irable and certainly more helpful in providing feedback for program 
improvement purposes. 

Tne niajor i;hooOOi:iin,* of an ex post facto design, from the traditional 
ret^earch point of vie>/, is the t^rave difficulty one encounters in trying to 
ascertain program effects without the benefit of experiment or quasi-ex- 
perini^intal con-.rols. Thus, the argument runs, ho\-r can one be sure that ob- 
ser\^ed outcomes of a program are due to the program itself and not to some 
extraneous fact;or(s) unless one makes pre- and post- treatment observations 
on both e:q)erimental and control groups? 

\'Je concur that it is impossible to determine cause and effect rela- 
tionships in the absence of rigorous experimental controls. But, as V/eiss 
and Kein point out, attempts to apply experimental designs in the evalua- 
tion of conplex social programs usually prove futile because such programs 
are constitutionally resistant to the requirements of precise measurement 
and experimental manipulation.-^ Furthermore, experimental or quasi-experi- 
mental designs, which necessarily emphasize input and output measures, pro- 
vide little information about the process of program planning and imple- 
mentation. It does little good to know that a program has succeeded or 

^•Robert VJeiss and Martin Rein, "The Revaluation of Broad-Aim Programs; 
A Cautionary Case and Mora]," The Annals of the American Academy of Political 
and Social Science , Vol. 385, 1909. 



failtKl '^UjiouL kiiuVirui why. 

In our viev/, it is necessary Lo sliiffc the tcrtas of the discussion and 
to al:andon t!ie cliiiaera of oauije aad effect in evaluating complex educational 
pro.$ram3. If we assume that the purpose of the evaluation is to provide 
information for projrari inproverr ^nt purposes (so-called formative evalua- 
tion), then the key question is: To what extent is the procram accomplishing 
what it seriously intends to accomplis}i? It follov/s, therefore, that the 
evalaahor (l) first establislies program intent; (2) subsequently collects 
(lata to determine current practice; and (3) compares data on intent with 
data on current practice to ascertain the natui-e and extent of any dis- 
crepancies. Should the evaluation reveal major discrepancies, those re- 
sponsible for tlie program can modify current practice and/or adjust intents 
in order to close the gaps. 

This approach to program evaluation, often termed "discrepancy analysis," 
vras articulated in its basic outlines by Robert Stake of the University of 
Illinois in 196?.'*" Tlie present investigators have employed discrepancy 
analysis in evaluating adult basic education programs.^ 

O/erview of !i>/alT:af-ion lYocediires 

Subsequent sections of this report describe the Columbia University 
Health Consumer P'ducation Program, present data on intended and current 
practice, analyze discrepancies, and discuss recommendations. Sources of 
data included interviews with key CEP staff, intervievrs with community 

^Robert Stake, "The Countenance of Kducational Evaluation," Teachers 
Colle.'je Record , Vol. 68, April I967. 

'^See Alan B. Knox, -/ack Mczirow, ':ordon ^1. Darkenv/ald, and Ifarold 
Beder, An Evalnation \vAdrt for Adult lasic Kducatlon Pro^rrams , V/ashington, 
D,C.; U.S. iovernment irinting Ot'fice, 197^^. 



parMoLparit ij who cov rod aa oourtJe coordinator g, cyalualion s'aeeLs completed 
by all participants for every cource, a questionnaire mailed to all coarse 
participants, observation of a small number of classes, and analysis of 
doouments such as the ori^^inal proposal, progress reports, and course 
outlines. 

Intent was established by intervievrin^j the project director and the 
U/o assistant directors. They vrei-e asked to describe intended outcomes, 
riOt necessarily as set forth in the initial proposal, but rather as they 
had evolved throu^^h experience and reality testing. Similarly, they v^ei^e 
queried about intentions related to program process factors such as extent 
and nature of participant responsibility for coiu^se planning. Reference 
will be made to intent as reflected in the official troposal, but the 
criterion of judgment is intent as determined by consensus of the three 
key staff members. It mi>:;ht be noted that there are often differences in 
intent among major actors in a pi*ogro.m, and that such differences, which 
may be the source of serious problems are normally brought to light in dis- 
crepancy evaluation. In the present case, however, there were no signifi- 
cant differences in intent among the three program administrators. It 
should be noted, too, that the intents of community residents who helped 
plan and implement the coiu'ses were not taken into account in this evalua- 
tion. The reason for this omission is that involvement of any one coimnunity 
resident v/as limited to particular courses in a particular neighborhood. 
Our task was to evaluate the total program, which ultimately was the re- 
sponsibility of the CEP staff. 

Current practice, vrith emphasis on program process and outcomes, was 
assessed mainly through interviews vrith course coordinators in each of the 
target communities and through a questionnaire survey (see Appendix A) of 
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'Ire course, ooorJinalorG vrero a vory in:porLanl., aonrce of inronralion 
;ihouL CHF. About a dM^ioii ^:o'Pfiumit.y renUlouts in Uic t;;o-.ye:ir period served 
as Q'vjrdlna' ors in Lhe fouv lar^el 'jotmuunities. Vlie coordinators vz-ero chosen 
by the course sl.eerin,^ co^uuittees, uliich consisted of !ioca3 residents wiio 
volu>itoered to or\/anize a>Kl plan the courses. In efrect, the local coordi- 
nator was ch:?Jrf'ian of the steering eoiruuittee and hoA the majoi^ vole in actual- 
Iv .(v,>riductin:5 the classes, e.g., locatin,; space to meet, contacting potential 
faculty, inuroducin^ speakers, ^'our coordinators and one steering committee 
I'lCHioor ;^ere seleeted for in-depth, open-ended inter^/ie^/zs. IVo v/'ere selected 
frcm the 'larlera coainiunity (rnuch of the action vms in Ilarlem) and one each from 
Central Brooklyn and Jllliaius-xirj-^eenpoint • The steering coaimittee member 
participated in the proo^m conducted nn the LoTOr East Side. 

[♦bllovanij interviews vn.th CEP staff, examination of the project proposal 
and '^ther documents, and observation of classes in Ilarlem, a brief question- 
naire v/as de/eloped (see Appendix A) and mailed to all 33- CKP participants 
with a cover letter (Appendix B) :b'i-;ned by the CEP staff. After a follow-up 
appeal, usable returns v/'ere received from 115 participants, 3^+^^ percent of 
the total. 

The response rate was less than we had hoped to achieve, but about 
average for a mail questionnaire, ly.-ro factors appeared to be of particular 
si,;aificance 1n suppressing the response rate. Many participants in CEP 
were poor and undereducated. Our own observations, confirmed by the staff, 
su^,^^ested that a significant percenta.ie were barely literate. Inspection of 
oDen-ended responses from those v;'ho returned questionnaires indicated that 
many could write only with great difficulty. Although the questionnaire 
was deliberately designed to mitigate this problem (it vras brief and 
simply worded), there is little doubt that a large number of CKP partici- 
pant's could complete tlie instrument only with great difficulty, if at all. 

; ; 8 
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Poor pooplo als'j tend to be very mobile and diiTiculfc to reach by mail or 
telephone. Gixty-tvro ciuesiionnaires were returned as uudelivorable with 
cuch notations as "mo7cd--lef t no address," "addressee unKno\m," and "un- 
clairnod." Thus, v;e received responses from about h2 percent of tho/je who 
could be t^eached by mail. 

In order to estimate the probable direction of non-re;5ponse bias, the 
first wave returns (n=78) were compared on all variables with the second 
v/ave returns (ii=37). "I'he reasonin^^ behind this procedure is that those 
wiio are more difficult to reach (i.e., those vrho returned questionnaires 
only after a follow-up appeal) are more likely to possess certain charac- 
teristics in common v:ith non-respondents. We did find some diffei*ences. 
C 'Cond vrave respondents savr the conununity has having less say in desicning 
and runnin^j the courses than did their counterparts. Moreover, a larger 
proportion of the second vrave group indicated that CEP was of little help 
in improving their ability to "participate effectively in community health 
care decision making." Finally, there was a strong trend (but not statis- 
tically sli;^nificant ) for second wave respondents to indicate less involve- 
ment in community health affairs as a result of participation in CEP, 
Taken as a vrhole, these differences suggest that the second wave group, 
and by extrapolatioii the non-respondents, tended to be somewhat more 
skeptical about CEP— or perhaps critical of the University's influence in 
the conmiunity. Hovrever, on every variable not mentioned above, including 
learning gains and recommendation of the course to others, no difference 
was found between first and second v/ave respondents. Perhaps the non- 
respondents were generally negative for one reason or another. In any event, 
generalizations based on a 35 percent response rate must be considered 
tentative because of the possibility of significant non-response bias. 
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Tlvj lu'ilu p;iri)oco oV Iho vino:;LLonnalro v/as *:o onl/aln dai.a from partici" 
panL:T al.ouh LlKjii'pcrcep-^ion of VS.? -i^yy-i'iiir, (o.j., how courses wove planned 
and u'lplcn.ontjil) and out >vv>..;r> (kno'./lod.^o and skilL'^, involv^Mr.oul in com- 
rumloj l.oalhl> caro activity). After co.Unrj of opeu-onded ilAViits, the 
quoo'tioiir.aire data vere ke^.puncliod and vorified and frequency and percentile 
di:>trLbi;tion3 for each variable v/ere -^^'^'-d by computer. Further analy- 
sis consisted of two-vjay crojs-iabulatlons betvceu selected variables such 
as reason for enrolling and extent of subsequent participation in community 
'.lOaith care decision making. Tlae Hhi Square statistic was used to determine 
statistical significance of the cross- tabula ted data, with alpha set at 
.05. 

"eeause tiio study did not be'j;in until the last month of C^'P's grant 
period, it was not possible for the evaluators to systematically observe 
a reproscn native saiaple of classes. Direct obser\at,ion i.s a valuable 
strate:^y in process oriented evaluations, but che exi^^encies of time and 
money precluded its use in the present case. Lack of opportunity for 
sys^ie^'iatic observation v:as, in our judgment, an unfortunate and serious 
limitation of the present study. 

Poser ipt ion of tbe Col;im b i a Kealth 
C'^ S'vr'.M' :.du(^aviori i ro^^ram 

Columbia University's flealth Consumer Education Project has gone 
aoout its task of providin^^ education for community residents v/ithin the 
context of a politically ciiar(-;ed and hi,.;i:ly complex netv/ork of a^^encies 
and ori^anizations representing health consumers and New York City's health 
planning bureaucracy (known as the B A^jency). The politics of comprehensive 
health planning (CrIP) are well beyond the ken of this evaluation. It is 
germane to note, however, that the political context of consumer participation 
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in CiiP aJvcrJoly alToclcd Iho tlaiverjity' s ability to mount and ^sustain 
cft'ecLive programs tarjoLod on actAial health decision makers. 'Hie ori^iinal 
^rant, proposal callod for oducaLion of consumer and provider members of 
local CHP boards, of which th.ero wore to be 33 in the city of New York. 
The idea was to help those local iiealtli decision mokers (especially the 
consumer layuen) to effeCui^/ely discharge their responsibilities by pro- 
viding education related to health problems and Ihe health care system and 
to the ro.3poii^:ibillties and tasks of board membership, i^ut ^;heti the pro- 
ject. ,;ot underway in the summer of 1972 the local CilP boards had yet to be 
^ T.med. Ti*: fac". New York City's B Agency did not desj^^nate any local CUP 
Ljards until January, 197^, six months before the CEP prOeject v/as scheduled 
to terminate. As a result of these dovelopm-jnts, the ClOP staff was 
obli^^ed to sluft the focuo of the pro,;ram to pru/ide consumer education to 
local cominvmiLy recidcritj:5 wlio wore interested in or involved in health 
care matters and who ml;! it at some later date become members of a local 
CHP board. 

Tlie past two years have been marked by controversy between the B 
Ar;ency and various community health groups about ohe composition and powers 
of the yet-to-be-named CIIP boards. There has also been conflict among the 
various health groups and agencies in the community. There is a widespread 
belief amon^^ community health activists that the B Agency lias no intention 
of delegatiri^^ real power to local CllP boards. One consequence of the 
pervasive strife is that CHP often found itself in the middle of the 
cross-fire. 

The or^^anization and functioning of the actual program is described 
here only in its barest outlines to provide the reader with a frame of 
reference for interpreting the evaluation findings and recommendations. 
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A dcUxiloi 'ioscript.ion of Cl^?' r> work j ii ibrlom, co-auMiurod by CKP staff 
and local community pnrtioipnnh.s, can be found in a recent paper. 

The C/.P sLaff consisted of three proL*ossional;3, two of whom dovoLed 
full tiine to the project, Tne tin rd, the project dirce'.or, wa.s also direc- 
tor of Continuing Ivlu^jation in the ;:^chool of Public Ifea] Ih and had ot.her 
rorjponsiiu] Lties in addition to CEP, All tiiree staff members had had some 
experience in connuuuty development \:ork. One assistant director was 
trained in social work, the other in public h»oalth. The director had a 
Laokf^^roiuid in public health and achilt education* 

The ci^:nmunity development and adult education backgrounds of the staff 
proved hi>;hly siciiifi^'ant for the direction t'ne project took. A cardinal 
principle of adult education theory, v;hich is seldom found in practice, is 
that adults sliould take major responsibility for their own learning. It 
follov/s from this theory that the "teacher" doos not frame lesson plans 
and instrii'^t students, but rather serves as a resource person in assisting 
Mie learners to plan and iir.plefp.*'f^t tiieir own educational activities. It 
is assumed that adults learn best not as passive absorbers of knowledge, but 
as active a^^ents who take the initiative for the design and implementation 
of their o^^i learnia^ e;<periences. Aside from assumptions based on adult 
education theory, this approach had at least two very pragmatic advantages 
in the case of CEP. First, it v;ould not have been possible for Columbia 
University to go into a community such as Marlem and simply announce that 
a course in health education would be given and distribute the syllabus. 
If the University was to have any hope of success, it would have to work 

^!4\rcia i!el]er, al. "Consumer Planned Kducation in Comprehensive 
Health Planning.** I'apor presented at the annual meeting of the American 
Public Health Association, Uan Francisco, November, 1973» 

o 12 
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collaboratively with t:ho communily to assess learning needs and plan and 
implcnieni prot^rams to meet those needs, ttoroover, how wuld the staff of 
CEP knov v/l;at to ir.cliido in a prc-dotermiiicd course syllabus? A second 
benefit of the "adult edacation approach" was that the program itself pro- 
vided a model and learning experiences tliat were directly related to the 
ultimate objective of involving coniniunity rosideuts in assessing cotrmuuity 
health care needs and in plannin.'j for more effective health care delivery 
systems. The latent lesson vras this: "If you can plan and i^iplement a 
health education pro/sram for uhe cominuniby^ then surely you will be able 
to plan and implement programs for improvin,5 conimunity health care." 

Clearly, then, the educational process was every bit as important as 
the content of what ^,ras learned. Perhaps the process was more important, 
since the ^oal of CKF vras not to produce experts in public health, but to 
encoura^^e local residents to become informed participants in community 
Itealth care decision making. 

CSP worked in four lov; income communities where there v/as grasq#*oots 
interest in health problems and where the University v;as invited by involved 
con'jnunLty residents to provide assistance. T\vo communities, Central Harlem 
and the Lower East Side, were located in Nianhattan. The other two^ Central 
3rooklyn-.Crown Heights and Williamsburg- Jreenpo in t, vrere located in Brooklyn. 
Because of its proximity to Columbia University and because of the communi- 
ty' s previous contacts with the school of Riblic Health, CEP started its 
first courses in Harlem in fall 1972 and continued to TOrk there until 
September, 197^. 

Of the l8 courses and workshops supported between 1972 and 197^^> nine 
were civen in Central Harlem, five in Will iamsburg-Greenpoint, three in the 
Lower East Side, and one in Central Brooklyn-Crown Height?. Tlie total 

ER?C 13 
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iiumbov oV ;;omra^c iiiMiv i diuil.^ vlio p''.r r LelpaLod ':as 33^^ Of these, a jjI^- 
r.ifieant p?'oporlion (^3*3/j) i.ook taovo than one course* 

ri\e prucess oT pro^^r-^m doYciopcior^t was similar in each comjaunily. 
After contact had been os'-abHsh.od between C'S.? ivid ' tores tod ccmtmiulty 
groups ( 'sually at the initiative of the comiunity), a (;jeperal meeting of 
eoiiiiTtriily rosiden\rj waj ^M'.dely publicized (throU(jh notices, flyers sent to 
local or^atiir.ations, etc.) to discuss local health care problems and the 
i:v}ed for community health, education. At the open meeting the general pur- 
pose of the C:OP project was explained, the comjiiunlty >ras invited to react, 
ard finally, atMer a period of discussion, a^^reoinent was reached on the 
ueed for CA'niiiunLty h.oalth education courses. An important outcome of the 
ox;eu meeLLi.-^- was th.e formation of a "steerin^^ committee" of community 
vol*;nteers to undertake the task of assessing learning needs and objectives 
and planning a course for community residents. 

The course steering coinmittee \rj.s a crucial component of CEP' s com- 
munity education strategy. In each case, the local commitiee, with the 
aid of C;';P staff, made the major decisions about course goo.ls, topics for 
individual sessions, and recruitment of learners and faculty members. 
Most steerln; coiimilttees consisted of six to eight members, although one in 
Harlem had only tvro members, wh'le another in the Lo^^cr Kast Side had 1^+. 
The plannin.;; process typically required about six v^eeks, with the average 
committee meeting once a week in tv^o to three hour sessions. Follovaug 
the planning period, usually at its last meeting, the steering committee 
designated one of its members to serve as course coordinator. As noted 
earlier, the coordinator vras responsibe for the week to week conduct of 
the course, e.g., contacting prospective faculty, introducing speakers, 
serving as discussion leader. 
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According bo CKV stat^f, hhc cumuiiit:/ sLceriii^ conimibLees vrere respon- 
sible for ail decisions coucerninj the couruos, the CEP providing advice 
or assistance v/hcn asked. There v:a3 a reported tendency for coinraunity 
groups to be vrary of the University initially and to rely minimally on CEP 
for assistance. Later, wl^eu greater trust developed, CEP staff had to 
resist a tendency for steering committees to turn over their responsibili- 
ties to the university "e>::perts." 

'^'he products of the local steering committees efforts were a variety 
of courses aad workshops uith widely divergent objectives and learning 
formats, and of varying in.tensity and duration. (The course outlines are 
f'jund in Appendix C. ) VJeekend v/'orkshops on such topics as nutrition and 
proposal ;^i*iting tended to be higiily focused and logically structured. 
Some of the lon-^er conventional courses v/ere also characterized by a single 
underlying theme, such as the l7jlliamsburg-i)reenpoint course dealing ^vlth 
tiie organization and adtninistration of health care systems (sample topics: 
Administrative structure of Health Agencies; Hospital Structure; the 
Medicaid System). Other courses, such as the 19 session Central Brooklyn- 
Cro^m rieights course of December *73 to February '7^, consisted of a 
smorgasbord of topics '>rith no discernible rationale or structure, e.g.. 
Communication Skills, IVnergency Health Services, Patient's Rights, Drug 
A.b'ise, Nutrition, Pi*eventative Medicine. A novel program of three special 
vrorkshops :/as labelled '^'lYain the Trainers." The purpose of these in- 
tensive courses was to train community residents in the skin?? needed to 
do what CSP was doing, namely to serve as resource persons or teacliers for 
community health education and health improvement efforts. 

It was mentioned above that the steering committees xvere responsible 
uuL only for planning the courses, but for selecting faculty and recruiting 
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"st'.idGuis." Accordln,^ to C7,V isl<xt^\\ Mic coniiniLteoij' main criterion in 
select, Li-; faculty was to "jel the best perjjon possible/' Cor a particular 
topic. Faculty mo::ibers tended not to be Academics, but representatives 
of healt'p. consumer advocacy ajcncies or spokesmen for various se^jinonts of 
the health care planniiic^ and delivery (provider) establishment. Over time, 
a core rroup of a dozen faculty resource people ernerged, mostly individuals 
who were lively teach.ers and "really kr.evr their thing." Only one of these 
"re.^julars" held a university appointment. Tnc others represented such 
diverse or.j;anizations as the Nev York State Healtli Planning Commission and 
Mobilization for Youth Legal Services. 

Students v/ere recruited in a variety of t/ays, sometimes with great 
vi^jor, sometimes not. Probably tlie single most important soui*oe of parti- 
cipants were community a.jencies and j^roups concerned in some v/ay with 
health care. About 32;^ of the students were providers, i.e., employees 
of hospitals and other health care agencies; the rnajority, however, were 
consumers, local residents with an interest in health problems. According 
to C:::? staff, participants tended to be active in community organizations 
and t^/pically were middle aged women, middle class in outlook, and of 
minority racial or ethnic background. Our ovm limited classroom observa- 
tions supported this characterization of the "typical" participant. It 
should be stressed, however, that participant backgrounds varied consider- 
ably, especially outside Harlem. 

CEF's community development approach necessarily resulted in the 
creation of a broad range of learning experiences reflecting needs and 
priorities as defined by each of the four local conununities. CEP, then, 
can best be viewed not as a unitary program of consumer health education, 
but as a process of community education which resulted in a variety of 
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di^lercnb pr')^5ra!!lc to hoperully tncet the needs of different coimiiuriities. 
The cormnon thread in all this activity was the overall objective of 
helping adults bo be informed and active participants in conimiuiity health 
care decision making. 

Intended Practice: I^Oi^ram Goals 

This section recapitulates program intent: what CEP seriously and 
realistically intended to accomplish. The above description of the pro- 
gram, v^hich vras based on project documents and interviews v/ith CI-lP staff, 
also illuminates intended practice. But in this section we define more 
explicitly the major program goals articulated by CEP staff that served 
as the basis for discrepancy analysis. 

A general systems framework provides a uset\il way of ordering program 
intents, vor most educational programs, the basic inputs are participants, 
faculty, and other reso\.rcos such as facilities. The throughput consists 
of some sort of educational process which transforms inputs into outputs. 
Output is generally enhanced knowledge and skills and perhaps their appli- 
cation in problem solving situations. 

Interviews with CEP staff re7ea\ed that the overriding intended out- 
put was a corps of participants who were aware of and knowledgeable about 
coimnunity health care problems and issues and who had become actively in- 
volved in applying their learning to improving health care in their com- 
munities. Successful attainment of this objective was intended in the 
long run to produce health consumer leadership and a community based con- 
sumer "constituency" for this leadership. 

These output objectives were to be achieved through a process best 
described as community development through education. As noted previously, 
it was assumed that adults learn best when they take responsibility for 
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their own learniiij. Cont:equont]y, Iho two key process objectives i^rere 
active involvemoiit by coimuanity members in assessing community needs and 
desi^ninj programs and in implement in^.^ programs once they had been planned* 
By placing responsibility for establishing the classes in the hands of the 
participants it was hoped that they would have an opportunity to practice 
the organizing skills essential for cor.ununity action. The idea was to 
give the participants a sense of potency— to demonstrate to them that it 
vj-as their responsibility to achieve change and that they could indeed do 
so. 

More concretely, the instructional process was intended to: 

- increase avrareness of coirnnunLty health care problems and 
issues 

- enhance knowledge and understanding of community health 
problems and the nature of the health care delivery 
system 

- help participants identify alternative courses of action 
to improve health care in their communities 

- teach the skills needed for effective action to improve 
community health care* 

Intended program participants were opinion leaders and potential 
opini'^a leader's from low income communities plagued by health problems 
and inadequate health care services. It vras expected that some partici- 
pants would come from established cormnimity agencies interested in health 
care, that others would come from organizations that provide health care, 
and that a third group vrould be comprised of those without organizational 
affiliation who were concerned about health care issues and problems. 

The steering conmilttees were expected to take major responsibility • 
for selecting faculty, and thus CEP staff had no expectation of setting 
criteria to insure "high standards." Nevertheless, it was anticipated 
by CEP that faculty would be "the best available" in their area of expertise 
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and TOuld be able to rolale well or '*get across" ho the ooiMuniLy partici- 
pants. 

Key pri^rani inteuLs are summarized below in a very rough systems 
framevrork. 



our^'Mary of Pi-o^ ratn Intent 



Input 
JJtudents 



I'^aculty 



Intent 

Students should be opinion leaders and potential 
opinion loaiers fror^ co'nmunities pla,:^ued with 
health problems aad inadequate health care 
services. Taey should volunteer to participate. 

Faculty should be selected by the steering com- 
mittee, 3>iould be very knov/'ledc^eable of their 
subject nia?:ter, and should relabe vrell to the 
participants. 



t'rooess 



Program Planning; 
and ImplcT^entation 



Learning 



Comunity residents, through a course steering 
com^nittee and course coordinator, should take 
the rrtajor responsibility for assessing community 
needs, designing and implementing courses. 

The learning experience should aim at increased 
awareness of health care problems and issues, 
enhanced knov;ledge and understanding of community 
health problems and the nature of the health care 
system, ability to idencify alterna^jivc courses 
of action to improve health in the community, and 
skills needed for effective action to improve 
community health care. 



Output 

Change in 
Participants 



Participants should acquire the skills and know- 
ledge necessary to improve health care in th.eir 
communities. Further, they should become more 
involved in efforts to improve health care or 
am.eliorato health problems in their communities. 



19 



17 



AcUi'\l ri'a c Mco: 'Hio Survey Ko nults 

P rot'ile of Par'Ucipants 

Of the 115 rospondetits, 36' (31-3'/j) ^ere employed in health-related 
jobs, almost exclusively at the paraproCessional, clerical, or technical 
level. Half a dozen de»scribeci Iheuselves as conununity health workers or 
aides, several TOre technic iauG, clerks, and licensed practical nurses* 
One directed a family planning clinic, another a nursing home. A handful 
v/ere re^jisterod nurses. There were no physicians, hospital administrators, 
or public health officials aiaon^i the "provider'' participants. The "con- 
sular" participants seemed fairly representative of low income community 
residents. Several described tliemselves as coimiiunity organi^iers, others 
were school aides, bookkeepers, students, blue collar workers, and house- 
wives. 

'.^en asked to indicate their main reason for participation in CKP, 
about tvro-thirds responded "to i)\crease my knowledge of the health care 
system so I can work effectively to change or improve it." Approximately 
one-fourth checked "I was interested in health care probleiaS in my com- 
munity and wanted to learn more about the subject," Only I3 individuals 
said they attended because of jobs with "a community program, agency, or 
organization concerned v;ith health care." Participants employed in the 
health field were equally as likely as others to want to "change or 
iniprove" the system. 

About three-fifths of the respondents reported they had taken more 
than one course. Of this group, about half participated in two courses 
and half in three or more courses. Those not employed in the health 
field were equally as likely as providers to have taken two or more 
courses. 
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Porc(^ T) I i or iG of i>J-icaLiona l rrocess 

Participants v/oro asked: "How much t^txy did coiiimunity mejuber^ have 
in deciding what your Consumer Education Course would be like?" Of those 
vrlio responded to the iconi (iO^), 2Q'!o Indicated that coimnunity merabers had 
the o^^eabest say* About oO-o said the University and the community had 
about equal cay, and the remainder, about one in eight, said the University 
had the greatest say. 

A second, related question, asked: "\\Tio took most of the responsi- 
bility for or^atdzing and running the Consumer "rkiucat\on Coiu^ue?" The 
dissrioution of responses was similar* About 20^* indicated ccjiinunity; 
oO-o indicated equal responsibility betv.^een University and community; and 
'20-j indicated University* Employment in the health field, number of 
courses taken, and motivation for participation were not related to per- 
cep^ion3 concerning vvho had the most say in deciding what the cource vrould 
be like and who had greatest responsibility for organizing and running 
the course* In short, then, about eight out of ten respondents said that 
the community, or the community jointly v/ith the University, took the 
iiajor responsibility for planning and implementing the CEP courses they 
attended* 

Perceptions of 7iV icational Outcomes 

Participants vrere asked: "How much did you learn about community 
health problems as a result of your participation in the Consumer Education 
Program?" Only Q.7v indicated they learned little they didn't know before* 
32*2:;^; said they learned some things they didn't know before, and 59*1?& 
said tliey learned many things they didn't know before* 

A related question asked: "How much did you learn e^bout the organiza- 
tion and workings of the heai.th care delivery system in your community as 
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a result of the CouauMor Isducati'j):! Pru^ram?" W]!j indicated "little," 
33i "Gomo," and 55 "a .;reat deal." Not surprisingly, only horfo of those 
employed in the health field said they learned a great deal, compared with 
02.5/!'' or the consumer (^roup (x^==6.6, dr=2, p<1.05). Also not surprising- 
ly, a lar^;,er proportion of those vrho took tuo or more courses reported 
learning a ^reat deal {r>2.5i compared to hk.h<fo, x^^g.O, df-2, p<.02)* 

Respondents vore also asked: "To what de^^ree, if any, did the Con- 
sumer .education Pro^viraEn improve your ability to participate effectively 
in com'iuuity health care decision making?" About 13% reported that CEP 
lielped little; h(yfo reported it helped some; and ^5^^ reported it helped a 
^reat deal. There was a subs::antial difference on this item betv/een those 
who took only one course and those who took two or more courses. Only 
25.5%' or t.he former ^roup reported that the course "helped a great deal 
to improve my ability to participate effectively," compare! ' 1th 6l.5^ 
or the lai -^^r group (x^=^lU.lf, dr=:2, p<.00l). 

Respondents were asked whether, before they participated in CPIP, 
they v/ere "acti^/ely invo ved in, any community grcap, organization, or 
agency that was at least partly concerned with corrjnunicy health care 
problems." A second question asked if they vrere currently involved in 
any such group or organisation. 85. ^fo indicated involvement before par- 
ticipation in CKP and 78.2',"^ said they v;ere actively involved at the time 
they completed the questionnaire. Overall, then, participation in CEP 
appeared to have no effect on active membership in community health orga- 
ni^iations a large majority v/ere already involved before participating 
in CEP. IIo;fever, it is interesting to note that those employed in the 
health field reported greater participation after the course (91.HyO than 
did health consumers, of vrhom 71.2'j^ reported participation in health 
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organizations roiltr^in^ CKP (x^^ii.^, df^l^ p<.Or)). Moreover, Tyj.^/o of 
tho.se who took two or more courses reported participation in health 
organizations follov/ing CKP, compared vrith 67 Mo of those who took only 
one course {x^^k.h^ df=l, p<.05). 

Another question^ focused explicitly on application of kno;/ledge and 
skills -sained in CEP, asked: "As a result of your participation in the 
Consumer Education Frosram, have you become more involved in community 
health affairs?" A follow-up question asked those responding affirma- 
tively to "explain in a few words how or in what way you have become more 
involved in community health affairs." ^7. hi said they had become more 
ifivolvod in community health affairs because of CEP. Of those who took 
two or mjre courses, 7l/j responded affirmatively to this quesi,ion compared 
with only 39.1/^ of those v^o took one course (:<^=9.7, df^l, p<.Ol). 
Interestingly, those who said that they participated in CEP in order to 
"chan^^e or improve" the system did not report greater participation in 
comniunity health affairs than those vrho enrolled for other, perhaps less 
militant motives. Nor was greater participation related to emplo^Tuent 
in the health field. Of those who reported greater involvement as a re- 
sult of CEP, three-fourths gave some kind of relevant response to the 
question of "how or in vrhat vmy." Listed below are representative re- 
sponses to this question. 

- active in evaluating medical centers in community 

- encourage people in community to seek proper health care 

- attend more meetings in community 

- able to give more information on health problems as block worker 
• Joined community health group 

- joined two ambulatory care committees 
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- joined orr,anizalion ho (^ot funds for hoalUi education pro^jrams 

- joined llarlom ot.ocrin,^ Connnlilee 

- volunloorod to work in hospital 

- ran for coiniiumity board of local hospital 

- Joined hospital committee 

- work wi th drug programs 

- working for community dental services 

- joined hospital advisory board 

- orf^^anizin^^S mothers to help retarded kids 

- more effective in /health-relateOT' 3ob (mentioned several times) 

- became chairman of health orj^anization committee 

- trying to better health conditions in neighborhood 

- joined comprehensive dental healtli committee 

A final question related to application of knowledge and skills 
asked: "To what degree, if any, did your participation in the Health 
Consumer Elducation Program prepare you to teach others in your conununity 
about health care problems and issues?" ^bout a fourth indicated that they 
felt "well prepared to teach others"; two- thirds said tl-ey felt "somewhat 
prepared, but not vz-ell prepared;" and the remainder indicated that they 
did not feel "at all prepared to teach others." Interestingly, those 
who took two or more courses did not feel any better prepared to teach 
others than did those who took only one course. Moreover, participants 
in special "Train the Trainers* '.-workshops designed to train community 
health resource persons v/ere no more likely than others to report being 
"well prepared" to teach* 

One universal measure used to evaluate courses and programs is the 
question, "would you recommend it to others?" On this criterion, CEP 
comes off very well indeed. Only one respondent indicated he would not 
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recomraond the courso or co\u:se3 he took to others in the community. Seven 
indicated "maybe." Rvcryone else responded "yes." 

The final item on the questionnaire asked for suggestions about hov 
the program might be improved. About two-fifths made relevant comments 
of one kind or another. Several suggested more "field work" (e.g., trips 
to agencies, observation of meetings), and one individual recommended 
"community based projects for practical application of learnings." A 
number of respondents commented to the effect that CEP should make a 
greater effort to recruit people in the conmuiiity not involved in the 
health field. Others suggested that the program could benefit from wider, 
more vigorous publicity. 

Actual Practice: Interview Data 
The following description of actual practice is based on in-depth 
interviews ;/ith four course coordinators and one student who was very 
active in planning CKP courses. The student v/as interviewed in lieu of 
a fifth coordinator, because the coordinator for the course was unavail- 
able for an interview. Interviews generally lasted from one to two hours. 

For most course participants, tae initial contact with CEP came 
through the recruitment process. This process is best described as dif- 
fuse. Announcement of an initial meeting of those who might be inter- 
ested in community health problems was filtered through the community by 
notifying community agencies, by word-of -mouth, and, in some cases, through 
announcements in community newspapers and nevrsletters and by distributing 
flyers. It is interesting to note that, in the majority of cases, those, 
interviewed were not clear about how they had been recruited. They ob- 
viously had learned of the impending program, but they did not remember 
just how or where. 
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At the Initial orvu.i^-atioiuil mooting, tho CKP pro<;ram was explained, 
rnrticlpants vero told U:at tlioy woi:ld have tho major say in determining 
course content, and selooLintj instructors. They were also told that they 
were to select a course coordinator and a steering committee to plan for 
the courses. 

The soloction process for coordinators was a consensual one. No 
formal votes were taken. Persons who were vocal, had experience in com- 
munity ori^anJ */,ation work, knew something of the health care delivery 
n.vs^.em, and wore x/illin^ to serve were Generally drafted Tor the position. 
All tho coordinators Interviewed possessed most or all of the above 
cliaracter istics, and none were very clear about how they had become co- 
ordinators. The general feelin,:^ was "I just fell into it." 

Tlie participants th.empelves seemed to be a diverse group Some were 
from organizations which provided health care, some were from community 
af^cncies focucing on health eare, and some were merely concerned citizens. 
Tlie ratio varied from community to oommunity. 

Once a coordinator and a coiirse steering conunittee had been selected, 
and once a core of participants had been identified, the next step was to 
determine course content, establish meeting times and dates, and to select 
instructors. In this process, those with knowledge of and past experience 
with health care problems seemed to have the greatest say. Although the 
decision makinj process seemed to be firmly under the control of the 
participant steering comunittees, the input of the CEP staff apparently 
had greater weight Uian that of the steering cofnmittee members. This is 
understandable since the CHP staff v/ere more experienced and knowledge- 
able in the health care field. Even tliough the Columbia team's input of 
expertise was greater tlian that of the participants, the course coordinators 
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all seeined to feel that auLhority for decision making; rested with the 
participants. The fact that course topics were quite siniilar in each of 
the five comnmnities is further evidence of the magnitude of CEP staff 
input. 

Instructors for the courses were suggested by the steering cormnittee 
members and the CEP staff. It was the general consensus that the quality 
of instruction was excellent, though in each case there were instructors 
who were less \rell received than others. Instructors who were well re- 
ceived at one course site were often asked to instruct at other course 
sites. In every case the classes seemed to focuf. on the major hospital 
in the community. In the case of the Harlem courses, classes were actual- 
ly conducted in Harlem Hospital. 'P/ro topic areas which seemed to elicit 
particular praise were ^'communications'* and **proposal writing." 

As previously explained, the basic model employed by CEV might be 
described as conmiunity development through adult education. The idea 
was that interested students would be made more aware of health care 
problems and would be equipped with the knowledge and skills to solve 
thom. Awareness, knowledge and skills were to lead to action. All inter- 
viewed saw great benefits to this approach, especially in comparison with 
the confrontational-political approach which has been often employed in 
disadvantaged urban communities. Interviewee? felt that learning how 
health care organizations operate enabled comjnunity residents to work 
with health care providers toward common solutions rather than working 
at cross purposes. One respondent remarked, for example, "I feel yon 
don't destroy something you cannot rebuild. You have to work with in- 
stitutions, not against them." Another stated that the approach elimi- 
nated the "grinding wheel for people with axes." All those interviewed 
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sooriiod to feci that the pro^^ram did supply the kiiowle^i^'^e and skills 
noodod for comhuiiiity residents to /ork within the system for positive 
chan^je. 

Yet 5 thou^^Ji the basic approach received accolades from the respon- 
dents, only one could give concrete examples of organized involvement 
i^rith conanuait.y health issues that continued after the classes had terminated. 
In this case several course participants had banded together in an effort 
to cet an alternative slate of candidates elected to the community hospi- 
tal' s advisory board. To say, how. ^er, that there is little evidence of 
organized involvement is not to say there has been no involvement. Often, 
evidence of involveiuent takes time to marxifest itself and the courses 
liave ended only recently. Also, our survey data show strong evidence that 
there has been considerable individual Involvement with community health 
care problems and issues. 

rievei"^helcss, most respondents expressed a wish that there had been 
.../re organized action as a result of the courses. Three of the four felt 
that the recruitment process should have been more selective — that re- 
cruitmer^t should have been focused on conmiunity leaders predisposed to 
or^iianized action. Most in':erviewed also felt that there should have been 
a follow-up phase to t^^ CIIP program dbv^us ^'^hich action cro^ips would have 
been organized and supported in their activities. 

The program was supposed to be open to anyone vrho wished to partici- 
pate. All evidence seems to indicate th^^.t this intent was realized. Yet. 
as some of those intervievred pointed out, an open door approach, whatever 
the benefits, creates certain problems. Not all participants were highly 
committed to th.o learning objectives of the course. In one conununity, 
for example, a large number of paraprofessionals from the local hospital 
attended. The course coordinator from that community felt that these 
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persons v;^ere often a diiiruptivo inriuence bocause they aLLondod in the 
bolief tliat participo.Lion ;/ould iiaprons their Guporiorc and enhanco pro- 
motion pos:jibililies, not because they wished to improve health care in 
the cormnunity* Wlicther or not our respondent's assessment is warranted, 
it does seem probable that difforei:ces in the vay participants related 
to this particular course caused some tension amony students. It is 
likely, however, that the problems associated vdth this class vrere atypical. 

Open ended enrollment seemed to have produced considerable diversity 
amon^ students in regard to ed^icational lovul. Tills diversity made the 
instructor's r^ole somev;hat difficult in some cases. Our respondents, for 
example,- iri licated that in every course sequence there ^^ore instructors 
who either talked above or below them. hWf^rtlinless, the coordinators 
felt tiiat most instructors mana^^ed to gear their presentations so that 
i.hose at different educational levels could benefit. Talking above or 
below the majority of the class v/as the exception rather than the rule. 

T^yo of those interviewed participated in the Train the Trainer ses- 
sions. Ti\Q purpose of this component of CKP was to equip selected par- 
ticipants of the rOf^ular classes with the skills and knowledge to teach 
others what they had Icai-ned in the re^^ular classes and in the '[Vain the 
Trainer sessions. We v/ere unable to secure much information regarding 
Train the Trainers. One respondent saw considerable value in the work- 
shop she attended. Another felt that her particular Train the Trainer 
session, conducted at a weekend retreat, \ras not very vrorthwhile. 

In f^eneral, those interviewed assessed tlie CKP pro;^ram as very 
effective and beneficial. Tlie picture of current practice we received 
from interview respondents seems lar^jely consistent with intended practice 
as described earlier. 
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Comparison, of Tn^O!v!>-Ml/.;it]i Ackvnl IVactice 
As crplo,inod in Iho isection dealing; vrilh evaluation desi^^n, the basis 
for the present evaluation is a comparison betv,''eon program in Lents and 
actual program practice. Intent v/as established by interviews with CEP 
starf. Actual practice was established through our survey of partici- 
pants, examination of project documents, and through interviews with com- 
munity course coordinators. 

Intent 1 Students sliould be opinion leaders and potential 
'"•piiiion loaders from lo\r iw^ome coimnimities where the 
deli\/^ery of health care has been inadequate. Students 
should volunteer to parti pate. 

vie find no discrepancies in respcf^t. to this intent. Survey results 
Mvlicate that, prior to taking part in the program, 85. of our respon- 
dent:; were actively involved in organizations concerned with health care 
issues. It is reasonable to assume that a significant number of these 
persons were opinion leaders in respect to community health care and in 
other areas, and that a sii^nificant number of those remaining were po- 
tential opinion leaders. Intervicvr responses produced strong evidence 
tha'o all course coordinators ^vere opinion leaders vrithin their respective* 
cominunities. Recruitment \^ras open ended, and as a result CEP had little 
direct control over who v^as recruited. A measure of control vras achieved, 
hovrcver, by focusing initial publicity tnrough existing community agencies 
involved in or concerned vath health care. It is obvious that a majority 
of participants were recruited in this vray. All those who participated 
in CEP volunteered to do so. 

Courses were conducted in Harlem, the Lov/er I'last Side of iv!anhattan,. 
Williamsburg-Oreenpoint and Central Brooklyn. Each of these low income 
communities meets the criterion of inadequate community health care 
service 3. 
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Intent 2 , Faculty should be sQlected by the course steering 
coinialltee, should be very knov^ledijeable in their area of 
experLice, and iJhould relate woll to tlie participants. 

That program faculty generally were knowlGd{3eable in their areas of 
expertise and generally related well to participants is evidenced by the 
fact that ^2J^J of our survey respondents indiczited they would recoimnend 
the courGe(s) they took to others Ln their connuunities. Moreover, stu- 
dent ratin^js of each clastj session on "the way the guest speaker handled 
the session" were consistently positive for all courses in each of the 
four communities. The fact that general satisfaction was high, however, 
does not mean that there vras unconditional and universal praise for all 
faculty members. The educational level of course participants was quite 
diverse, ranging from functional illiteracy to college completion. Hence, 
while in some classes participants felt they v-rere being talked down to, in 
o':hers participants felt the material vras too difficult. In Oiir opinion, 
ho:/over, Lhe great majority of faculty members were apparently able to 
deal with the diversity problem, gearing their presentations to partici- 
pants of varying sophistication and educational attainment. Occasionally, 
community political leaders who were invited to address classes failed to 
show, and occasionally participants reacted negatively to what one inter- 
view respondent termed a "patronizing attitude." Yet, nearly all these 
problems were to be expected in a program such as CEP, and neai'ly all 
vrere caused by circumstances over x^rhich CEP had little or no control. 
Faculty who were ill-recieved were not invited to instruct other classes, 
and faculty who were very effective v;ere used many times and, in fact, 
came to constitute a "core" group of resource persons. 

Although it -.s probably safe to say that community members did in- 
deed retain the power to select faculty members, the process of selection 

er|c 31 



29 



was shared with CEP staff. The steering committee had the final say, 

but C2P staff had considerable influence because they kne;/ of, and had 

tjreater access to, a wider ran^e of faculty resources than did the steering 

conimittees. In no instance did we find that CEP staff input was considered 

illce^itimate or inappropriate. 

Intent 3 * Comniunity residents, through a course steering 
comnittoe, should take the major responsibility for assessing 
community needs and designing and implementing courses. 

We find little discrepancy between this intent and actual practice. 
The planning process began with an open meeting v/hich was publicized pri- 
marily by notifying community agencies active in the health field. In 
some cases, flyers were distributed and notices placed in community news- 
papers and agency newsletters. During the open meeting, CEP objectives 
were explained and those present were asked to form a coui'se steering 
committee which would select course topics and faculty, set the time and 
place for the coarse, and choose a course coordir^ator . In addition to 
giving the participants the major say in planning instruction, the 
steering committee was intended to legitimize the course in the eyes of 
the community. The planning process itself v;as rather informal. Course 
coordinators, for example, were selected by consensus rather thar by vote. 

To say that the steering committee and other participants had respon- 
sibility for planning and implementation is not to say tliat they were 
the sole decision makers. Decision making requires knowledge of alterna- 
tives and implementation generally requires resources. The CEP staff 
possejsed a certain expertise and certain resources that the steering 
committee did not. Consequently, their input to the decision making pro- 
cess was significant. About 28^0 of our sui'vey respondents indicated 
that the community had the greatest say in deciding what the Consumer 
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Education Courses would be like; 68^^ indicated that the University and 
community had about equal say, and the remainder felt that the University 
had the greatest say. CEP staff members mentioned that the steering com- 
mittees occasionally attempted to turn over responsibility to CEP, but 
these attempts were firmly resisted. 

Even though CEP staff input to the decision making process was some- 
times considerable, we uncovered no evidence that it was ever resented. 
In fact, one interview respondent criticized the CEP staff for not taking 
firm control of the decision making process. In our judgment, the role 
of the CK? staff in course planning and implementation could be best 
described as faciliiativet Leadership was exercised, and necessarily so, 
but always in the interest of supporting community-initiated effort. 

Intent The learning process should aim at increased 
av/areness of health care problems and issues, enhanced 
knowledge and understanding of community health problems 
and bhe nature of t-he health care system, ability to 
identify alternative courses of action to improve health 
in the community, and skills needed for effective action 
CO improve community health care. 

All evidence indicates that this intent was substantially rsalized. 
Examination of course descriptions shows that course subject matter was 
entirely consistent with the above intents. Generally, however, a par- 
ticular course addressed itself to only one or two of the above goals. 
Proposal writing, for example, was principally addressed to upgrading 
"skills needed for effective action to improve health care." Hence, if 
a student failed to take more than one course, he would not have the 
opportuiiity to acquire the full range of knowledge and skills listed 
above. Survey results indicate that hi. 7% of the participants took one 
course, 29.6% took two courses, and 28.7% took three or more courses. 
Therefore, more than half the total group were at least exposed to a wide 
range of learning experiences. 
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Intent 5 . Participants shouLl anquii^e the skills and 
knov;ied(je necessary to improve healLh care in their 
corrniunioies. Further, t.hey should become more involved 
in efforts to improve health care or ameliorate health 
problems in their coininunitles. 

Tt appears that, to a very considerable degree, these outcome intents 
were realized. We should note, however, that assessment of outcomes in an 
ex post facto evaluation is problematical. It cannot be asserted with 
complete confidence that the observed outcomes resulted solely or even 
mainly from the educational prof^ram. l^rthermore, our measures of learning 
ijains and behavioral change are based solely on participant self-reports. 
It v;as not possible for us to obtain pre-test/post-test scores to deter- 
mine the amount of knowledge actually acquired. We ore more confident^ 
however, about the reliability and validity of self -reports of "increased 
involvement." Participants either became more involved as a result of 
CEP or they did not. V/e see no reason why their reports should not be, on 
the \;hole, reasonably accurate. 

'I\^ro survey questions asked participants how much knowledge they had 
acquired as a result of CEP (knowledge of community health problems and 
knowledge of organization and workings of the health care system). Rough- 
ly, two-fifths said they had acquired a great deal of knowledge; only one 
in ten indicated they had learned little or nothing. Another question 
asked about improvement in "ability to participate effectively in community 
health decision making." Apparently, CEP was slightly less successful in 
achieving this intend-^d outcome. Fewer then half {k^i) reported that CEP 
"helped a great deal." Most of the remainder said CEP "helped some." In 
general, those who took two or more courses reported the greatest amount 
of learning. 

Perhaps the most crucial question was the following: "As a result 
of your participation in the Consumer Education Program, have you become 

34 



32 

more involvGd in community health affairs?" About half {^7*h%) responded 
that they had become more involved as a result of CEP. Tliis finding 
presents some ambi{];uities in interpretation. It may well be that many of 
those who did not respond affirmatively were already very .'^otively in- 
volved and could not reasonably become more involved. We suspect that 
this vac the case and that CEP v/as generally quite successful in achieving 
its primai-y goal of greater involvement by consumers in health care 
decision-making and community health care improvement activities. The 
quality, or meaningfulness, of this enhanced involvement was geii^-rally 
impressive. Responses to the open-ended item "how or ia what way have 
you become more involved" indicated a wide range of significant activites. 
These are listed on pages 20-21. 

Summary 

O/erall, we found a high de^^ree of congruence between program intents 
as articulated by CEP staff and actual practice as determined v»y a survey 
of participants and in-depth interviews with course coordinators from the 
four communities involved. The lack of significant discrepancies does 
not make for a very interesting discrepancy analysis, which is probably 
more of a "disappointment" for the evaluators than for CCP*s staff. 

Hot only do we find few discrepancies beLvfeen intended and actual 
practice, but we have little quarrel with the appropriateness of program 
intents as formulated by CEP. In general, we feel that Columbia University* 
Health Consumer Education Pi'ogram was soundly conceived and effectively 
implemented. It is heartening to find educators putting into practice in 
a competent, professional manner the tenets of adult education that are 
so widely endorsed in principle yet so seldom heeded in practice. 

Although CEP was quite successful, insofar as we were able to reliably 
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t:;au,ie success, we believe that CEP's experience raises some questions 
Mint merit further thou:ht and discassion. Accordingly, v;e off<ir the fol- 
lov;in,i; recoimTiendaiious to tjuide development of similar pro^-^rams in the 
future, whether at Columbia University or elsevrhere. 

Pecommendations 

1. Focus Recruitr^ent on Community Leaders 

CEP enco\ira{;ed anyone who was interested in health care problems and 
wanted to participate to do so. Although \re feel that this was an im- 
portant and commendable aspect of the program, we also feel that a more 
effective direct attack on health care problems could have resulted had 
the program recruited a greater number of participants who, by virtuPi of 
their demonstrated leadership in the community, could be expected to be 
predisposed to concerted action. Thus, while we would recommend that 
tlie program remain open to all (with perhaps less encoura^^ement of heavy 
provider representation), we would also recommend that there be greater 
effort to recruit community residents with demonstrated leadership abili- 
ties. Many of these individuals would have valuable ties to local com- 
munity action or^-anizations. The IVain the Trainers program was apparent- 
ly a move in this direction; perhaps it should have been given higher 
priority. 

2. rlmphasize Application of Skills and Knowledge 

CEP put a great deal of emphasis on the acquisition of knowledge 
through lectures and discussion, but relatively little emphasis on the 
application of knowledge and skills in problem solving situations. We 
recommend that ways bo devised to permit participants to apply knowledge 
and skills out in the community under the guidance of CEP staff or other 
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qualified profes;jionals» Students, for example, could be encouraged to 
participate in onjuin^j ooiirAinity health improvement projects. Periodically, 
they would be brought tojother for informal seininars to discuss their ex- 
periences and to probe tlie reasons for their successes and failures. 
Needed additional tralnin^i could be provided by CEP. If lack of success 
wore judged the result of improper application of skills or inadequate 
kno'Yledge, steps could be taken to remedy the problem. Several respon- 
dents indicated on the questionnaire that CEP could be improved by more 
attention to non-classroom learnin^^ experiences, such as visits to health 
care a^^encies and observation of CEP meetings. 

3 , ?co\' jrie Tech>iical Assi^itance for Community Action Projects 

In our opinion, CE? was quite successful in creating awareness of 
copramity health problems among its participants, and in some cases aware- 
ness led to direct action. We believe, however, that effective involve- 
ment and real change in community health conditions would be greatly en- 
hanced if CEP v;ere to add a technical assistance phase to its educational 
component. In the technical assistance phase, individuals and groups that 
decided to actively vrork for health care improvement would be supported 
in their efforts v/ith needed professional expertise. We are not saying 
that the University itself sliould organize direct action programs, but 
that the University should make its resoui^ces available to conununity 
groups that need assistance. In some cases, direct technical assistance 
could cause "political** problems for the University. Such problems might 
be minimized by channeling technical assistance through linkages with 
community health improvement agencies. Experience in community develop- 
ment has shown that education in itself is usually not sufficient to bring 
about fundamental institutional change. Some form of follow-up technical 
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assistance is almost always nccosb-ary# 

14-. Institute Contini;ou>^ lY^yram l^Valuation from Inception of Program 
After-the-fact evaluations, such as this one, have limited utility 
for improving programs; they also have built-in methodological short- 
comings. Ideally, evaluation should begin at the inception of the pro- 
gram ani should be geared to providing information to improve the pro- 
gram, to help redirect it, as it unfolds over time. The opportunity tliat 
continuous program evaluation provides for securing comprehensive, 
reliable and valid data arc obvious. Continous, formative evaluation is 
expensive, but if done well it is worth the price. 
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AFPEHDn A 



School of Fullic Health 
Columbia University 

CONSUMER SDUCAHON IN CQMmEHE?ISIVi3 HEALTH PIAIIMING 

tjUESTIONNAIRE 



Directions 



Please answer nil qucctions. Check only one box /V / for each question, or if 
you prefer, write the answcx* in your am words. Please don't be kind to us if 
we don't desei^ve it. Your criticisms will help us to do a better job in the 
future. Your answers will be kept confidential. Do not sign your name. 



1. How many Consumer Health Education 
courses did you participate in' 



/ / One course 

/ / Two courses 

/^37 Tliree or more courses 



2. H'ow much say did community members have / / Community members had the 
in deciding what your Consumer greatest say about what the 

Education Course would be like? course would be like 

(Check only one box) 

/ / Community members and Columbia 
University staff had about 
equal say 

/ 7 Columbia University staff had 
the greatest say 



3. Who took most of the responsibility for 
or^^anizing and running the Consumer 
Education Course? (Check only one box) 



/ / Community members took most of 
the responsibility for orga- 
nizing and running the course 

/ / Columbia University staff and 
community members took about 
equal responsibility 



Co]umbia University staff took 
most of the responsibility 



k. How mach did you- ' irn about community 
health problems l a result of your 
participation in the Consumer Education 
Program? (Check only one box) 



/ / I learned little that I didn't 
know before about health 
probleras 

/ / I learned some things I didn't 
know before about health 
problems 

learned many things I didn't 
know before about health 
problems 



- OVER - 
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. 2 . 



How much did you learn about the orga- 
nization and worki*-gs of the health 
care delivery system in your community 
as a result of the Consumer i^ducation 
Program? (Check only one box) 



rj 



I learned little I didn't know 
before about the health care 
delivery system 

I learned some things I didn't 

know before about the health 
care delivery system 

r7i learned a great deal that I 
didn't know before about the 
health care delivery system 



6* To what degree, if any, did the Consumer 
Education Program improve your ability 
to participate effectively in community 
health care decision making? (Check 
only one box) 



rn The program did little to im- ! 
prove my ability to participate} 
effectively in health care ;( 
decision miiking 

/ [Jt The Program helped some to im« 
prove my ability to participate 
effectively j 

/ / The Program helped a great deal 
to improve my ability to » 
participate effectively 



7* Befo re you first participated in the / 7 Yes 
Consumer Health Education Program, were 
you actively involved in any community I J No 
group, organization, or agency that was 
at least partly concerned with community 
health care problems? 

8. Are you actively involved now in any f 7 Yes 
community group, organization, or agencv 
that is at least partly concerned with I f ITo 
community health care problems? 



; 7 
.( ) 

I 
f 

Is 
i( ) 



9* Which one statement below best describes youv own reason for participating in \ 9 



^^^^ ^y^^^^ u^o^i xuca y jul' 'jwii rcasQn xor pa 

in the Health Consumer Education Program? (Check only one box) 

I vas interested in health care problems in my conxnunity and wnted 
to Ijarn more about the subject 

I attended because of my job with a community program, agency, or 
organization concerned v/ith health care 

£]T I wanted to increase my knowledge of the health care system so 
I could work effectively to change or improve it 

/ / Other reason (please describe) 



K ) 
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10, As a result of youf participation in the Health Consumer ZZ7 
Education Program, have you become Piore iri' r'olved in 

community health affairs? / / No 

11, If you answered Yes to '^estion 10, please explain in a few words how or in 
what way you have become more involved in community health affairs. 



12. To what degree, if any, did yo'or participation in the Health Consumer 

Education Program prepare you to teach others in your community about health 
care problems and issues? (Check only one box) 

/ / I feel well prepared to teach others in my community about health 
care problems and issues 

/ / I feel somewhat prepared, but not well prepared, to teach others in 
my community about health care problems and issues 

/ / I do not feel at all prepared to teach others in my community about 
health care problems and issues 



13. Would you recommend the course or courses you took / / Yes 

to others in the community? 



10 
( ) 

11 
( ) 



12 
( ) 
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/~7 Maybe ( ) 

/I/No 



lU. Please describe briefly the kind of work you usually do (for example: ik 
nurse's aide, machinist, housewife, accountant). 



15. If you have any suggestions about how the Health Consumer Education Program 15 
can be improved or if you would like to make any other comments, please 
write them below. ( ) 



THANK YOU '='0R YOrjR C00P2BATI0N 



APPENDH B 

COLLMIJIA L'MM KSII^ 

SCHOOL OF PUBLIC HHALTH of the Faculty of Medicine 



July 17, 1974 



Dear G>nsumer Education Course Parficipont: 



The Continuing Education staff of the Columbia University School of PObllc Health 
IS sponsoring a study of ttie Consumer Education in Comprehensive Health Pionring 
programs it co-sponsored with community groups in Harlem, The Lower Eost Side, 
Wiiliomsburg-Greenpoint and Central Brooklyn. 

The purpose of this study is to determine how successful the program has been tn meeting 
the needs of communities for consumer health education and to obtain information that 
will help us to improve our programs in the future. 

We hove enclosed a short questionnaire that should take only o few minutes to complete. 
We would appreciate it if you would fill it out and return it to us os soon as possible in 
the enclosed, stamped envelope. 

Yoiir onswers to the questionnaire will be kept completely confidential . You do not 
hove to sign your name. 

To do a really good study, we need the cooperation of everyone who participated In the 
courses. Please return the questionnaire today, if you can. We deeply appreciate your 
cooperation in this study . 



Sincerely, 

Marcia Pinkett Heller, M.P.H. 
Instructor, Health Administration 

Isaac Purdue 
ft-ojcct Assistant 
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